{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IxnZa5XULICe"
      },
      "source": [
        "Copyright 2021 Google LLC\n",
        "\n",
        "Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "you may not use this file except in compliance with the License.\n",
        "You may obtain a copy of the License at\n",
        "\n",
        "     https://www.apache.org/licenses/LICENSE-2.0\n",
        "\n",
        "Unless required by applicable law or agreed to in writing, software\n",
        "distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "See the License for the specific language governing permissions and\n",
        "limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HGjX15sjLNMQ"
      },
      "source": [
        "# Instructions:\n",
        "\n",
        "\n",
        "*   Public URLS are scoped by https://storage.googleapis.com/kip-datasets/ and can be downloaded directly from the browser \n",
        "while gsutil URLs are scoped by gs://kip-datasets and can be opened using tf.io.gfile.GFile. \n",
        "*   The KIP and LS datasets can be obtained using the URLs stored in csv files at gs://kip-datasets/kip/kip.csv and gs://kip-datasets/ls/ls.csv. Their contents are shown below. URLs point to datasets given by the parameters corresponding to the given row in the dataframe.\n",
        "* Datasets are stored as .npz files. They have keys 'images' and 'labels' pointing to the respective numpy arrays."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "executionInfo": {
          "elapsed": 5846,
          "status": "ok",
          "timestamp": 1642197636016,
          "user": {
            "displayName": "",
            "photoUrl": "",
            "userId": ""
          },
          "user_tz": 480
        },
        "id": "hDAjVIlVK3nl"
      },
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "import tensorflow as tf\n",
        "import numpy as np\n",
        "pd.set_option('expand_frame_repr', True)\n",
        "pd.set_option(\"display.max_rows\", 100)\n",
        "pd.set_option('display.max_colwidth',90)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vWYKi4NFLZqn"
      },
      "source": [
        "# Load KIP data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OTb4xleZLc7-"
      },
      "source": [
        "KIP data specified by columns in a dataframe\n",
        "\n",
        "*   architecture: ConvNet (the main 4 layer convolutional neural network with pooling used) and ConvNet3 (a 3 layer version); see nn_training.ipynb\n",
        "*   dataset: cifar10, cifar100, mnist, fashion_mnist, svhn_cropped\n",
        "*   ssize (support size): 10, 100, 500 for non CIFAR-100, and 100, 1000 for CIFAR-100\n",
        "*   zca: True or False (corresponding to 'zca' or 'nozca' in filename)\n",
        "*   l (label learning): True or False (corresponding to 'l' or 'nol' in filename)\n",
        "*   aug: True or False (corresponding to 'aug' or 'noaug' in filename)\n",
        "*   ckpt: 0, 1, 2, 4, 6, 8, 12, 18, 26, 37, 54, 78, 112, 162, 233, 335, 483, 695, 1000, and thereafter checkpoints in step size 500 up to the length of training (maximum of 50000)\n",
        "\n",
        "Remaining columns specify metadata: test acc, test_mse, and URL to npz file stored in GCS"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "height": 424
        },
        "executionInfo": {
          "elapsed": 313,
          "status": "ok",
          "timestamp": 1642197636488,
          "user": {
            "displayName": "",
            "photoUrl": "",
            "userId": ""
          },
          "user_tz": 480
        },
        "id": "vyC7lzJGLQl3",
        "outputId": "2d7d29de-6639-4a99-8924-565671ab29d8"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  \u003cdiv id=\"df-6600142d-444e-4bd6-a52e-cba7185d92cc\"\u003e\n",
              "    \u003cdiv class=\"colab-df-container\"\u003e\n",
              "      \u003cdiv\u003e\n",
              "\u003cstyle scoped\u003e\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "\u003c/style\u003e\n",
              "\u003ctable border=\"1\" class=\"dataframe\"\u003e\n",
              "  \u003cthead\u003e\n",
              "    \u003ctr style=\"text-align: right;\"\u003e\n",
              "      \u003cth\u003e\u003c/th\u003e\n",
              "      \u003cth\u003earchitecture\u003c/th\u003e\n",
              "      \u003cth\u003edataset\u003c/th\u003e\n",
              "      \u003cth\u003essize\u003c/th\u003e\n",
              "      \u003cth\u003ezca\u003c/th\u003e\n",
              "      \u003cth\u003el\u003c/th\u003e\n",
              "      \u003cth\u003eaug\u003c/th\u003e\n",
              "      \u003cth\u003eckpt\u003c/th\u003e\n",
              "      \u003cth\u003etest_acc\u003c/th\u003e\n",
              "      \u003cth\u003etest_mse\u003c/th\u003e\n",
              "      \u003cth\u003eURL\u003c/th\u003e\n",
              "    \u003c/tr\u003e\n",
              "  \u003c/thead\u003e\n",
              "  \u003ctbody\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e0\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar10\u003c/td\u003e\n",
              "      \u003ctd\u003e10\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.1648\u003c/td\u003e\n",
              "      \u003ctd\u003e0.047982\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/kip/cifar10/ConvNet_ssize10_zca_l_aug_ck...\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e1\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar10\u003c/td\u003e\n",
              "      \u003ctd\u003e10\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003e1\u003c/td\u003e\n",
              "      \u003ctd\u003e0.1806\u003c/td\u003e\n",
              "      \u003ctd\u003e0.045354\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/kip/cifar10/ConvNet_ssize10_zca_l_aug_ck...\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e2\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar10\u003c/td\u003e\n",
              "      \u003ctd\u003e10\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003e2\u003c/td\u003e\n",
              "      \u003ctd\u003e0.1943\u003c/td\u003e\n",
              "      \u003ctd\u003e0.044273\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/kip/cifar10/ConvNet_ssize10_zca_l_aug_ck...\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e3\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar10\u003c/td\u003e\n",
              "      \u003ctd\u003e10\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003e4\u003c/td\u003e\n",
              "      \u003ctd\u003e0.2253\u003c/td\u003e\n",
              "      \u003ctd\u003e0.043009\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/kip/cifar10/ConvNet_ssize10_zca_l_aug_ck...\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e4\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar10\u003c/td\u003e\n",
              "      \u003ctd\u003e10\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003e6\u003c/td\u003e\n",
              "      \u003ctd\u003e0.2726\u003c/td\u003e\n",
              "      \u003ctd\u003e0.042291\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/kip/cifar10/ConvNet_ssize10_zca_l_aug_ck...\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e...\u003c/th\u003e\n",
              "      \u003ctd\u003e...\u003c/td\u003e\n",
              "      \u003ctd\u003e...\u003c/td\u003e\n",
              "      \u003ctd\u003e...\u003c/td\u003e\n",
              "      \u003ctd\u003e...\u003c/td\u003e\n",
              "      \u003ctd\u003e...\u003c/td\u003e\n",
              "      \u003ctd\u003e...\u003c/td\u003e\n",
              "      \u003ctd\u003e...\u003c/td\u003e\n",
              "      \u003ctd\u003e...\u003c/td\u003e\n",
              "      \u003ctd\u003e...\u003c/td\u003e\n",
              "      \u003ctd\u003e...\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e12731\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet3\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar10\u003c/td\u003e\n",
              "      \u003ctd\u003e500\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e48000\u003c/td\u003e\n",
              "      \u003ctd\u003e0.6658\u003c/td\u003e\n",
              "      \u003ctd\u003e0.027963\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/kip/cifar10/ConvNet3_ssize500_nozca_nol_...\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e12732\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet3\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar10\u003c/td\u003e\n",
              "      \u003ctd\u003e500\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e48500\u003c/td\u003e\n",
              "      \u003ctd\u003e0.6725\u003c/td\u003e\n",
              "      \u003ctd\u003e0.027906\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/kip/cifar10/ConvNet3_ssize500_nozca_nol_...\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e12733\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet3\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar10\u003c/td\u003e\n",
              "      \u003ctd\u003e500\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e49000\u003c/td\u003e\n",
              "      \u003ctd\u003e0.6694\u003c/td\u003e\n",
              "      \u003ctd\u003e0.027889\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/kip/cifar10/ConvNet3_ssize500_nozca_nol_...\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e12734\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet3\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar10\u003c/td\u003e\n",
              "      \u003ctd\u003e500\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e49500\u003c/td\u003e\n",
              "      \u003ctd\u003e0.6637\u003c/td\u003e\n",
              "      \u003ctd\u003e0.027951\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/kip/cifar10/ConvNet3_ssize500_nozca_nol_...\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e12735\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet3\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar10\u003c/td\u003e\n",
              "      \u003ctd\u003e500\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e50000\u003c/td\u003e\n",
              "      \u003ctd\u003e0.6710\u003c/td\u003e\n",
              "      \u003ctd\u003e0.027936\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/kip/cifar10/ConvNet3_ssize500_nozca_nol_...\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "  \u003c/tbody\u003e\n",
              "\u003c/table\u003e\n",
              "\u003cp\u003e12736 rows × 10 columns\u003c/p\u003e\n",
              "\u003c/div\u003e\n",
              "      \u003cbutton class=\"colab-df-convert\" onclick=\"convertToInteractive('df-6600142d-444e-4bd6-a52e-cba7185d92cc')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\"\u003e\n",
              "        \n",
              "  \u003csvg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\"\u003e\n",
              "    \u003cpath d=\"M0 0h24v24H0V0z\" fill=\"none\"/\u003e\n",
              "    \u003cpath d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/\u003e\u003cpath d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/\u003e\n",
              "  \u003c/svg\u003e\n",
              "      \u003c/button\u003e\n",
              "      \n",
              "  \u003cstyle\u003e\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  \u003c/style\u003e\n",
              "\n",
              "      \u003cscript\u003e\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-6600142d-444e-4bd6-a52e-cba7185d92cc button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-6600142d-444e-4bd6-a52e-cba7185d92cc');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '\u003ca target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb\u003edata table notebook\u003c/a\u003e'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      \u003c/script\u003e\n",
              "    \u003c/div\u003e\n",
              "  \u003c/div\u003e\n",
              "  "
            ],
            "text/plain": [
              "      architecture  ...                                                                                        URL\n",
              "0          ConvNet  ...  https://storage.cloud.google.com/kip-datasets/kip/cifar10/ConvNet_ssize10_zca_l_aug_ck...\n",
              "1          ConvNet  ...  https://storage.cloud.google.com/kip-datasets/kip/cifar10/ConvNet_ssize10_zca_l_aug_ck...\n",
              "2          ConvNet  ...  https://storage.cloud.google.com/kip-datasets/kip/cifar10/ConvNet_ssize10_zca_l_aug_ck...\n",
              "3          ConvNet  ...  https://storage.cloud.google.com/kip-datasets/kip/cifar10/ConvNet_ssize10_zca_l_aug_ck...\n",
              "4          ConvNet  ...  https://storage.cloud.google.com/kip-datasets/kip/cifar10/ConvNet_ssize10_zca_l_aug_ck...\n",
              "...            ...  ...                                                                                        ...\n",
              "12731     ConvNet3  ...  https://storage.cloud.google.com/kip-datasets/kip/cifar10/ConvNet3_ssize500_nozca_nol_...\n",
              "12732     ConvNet3  ...  https://storage.cloud.google.com/kip-datasets/kip/cifar10/ConvNet3_ssize500_nozca_nol_...\n",
              "12733     ConvNet3  ...  https://storage.cloud.google.com/kip-datasets/kip/cifar10/ConvNet3_ssize500_nozca_nol_...\n",
              "12734     ConvNet3  ...  https://storage.cloud.google.com/kip-datasets/kip/cifar10/ConvNet3_ssize500_nozca_nol_...\n",
              "12735     ConvNet3  ...  https://storage.cloud.google.com/kip-datasets/kip/cifar10/ConvNet3_ssize500_nozca_nol_...\n",
              "\n",
              "[12736 rows x 10 columns]"
            ]
          },
          "execution_count": 3,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "with tf.io.gfile.GFile('gs://kip-datasets/kip/kip.csv') as f:\n",
        "  df = pd.read_csv(f)\n",
        "df"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a-6eSjIXLoln"
      },
      "source": [
        "# Load LS Data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PJmvqDhlLtIo"
      },
      "source": [
        "LS data specified by columns in a dataframe\n",
        "\n",
        "*   architecture: ConvNet (the main 4 layer convolutional neural network with pooling used) and ConvNet3 (a 3 layer version); see nn_training.ipynb\n",
        "*   dataset: cifar10, cifar100, mnist, fashion_mnist, svhn_cropped\n",
        "*   ssize (support size): 10, 100, 500 for non CIFAR-100, and 100, 500 for CIFAR-100\n",
        "*   zca: True, False\n",
        "\n",
        "Remaining columns specify metadata: test acc, test_mse, and URL to npz file stored in GCS"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "colab": {
          "height": 990
        },
        "executionInfo": {
          "elapsed": 219,
          "status": "ok",
          "timestamp": 1642197640744,
          "user": {
            "displayName": "",
            "photoUrl": "",
            "userId": ""
          },
          "user_tz": 480
        },
        "id": "a6xd_MgMLgMs",
        "outputId": "10b901ff-1685-4717-f05a-230f9426dde3"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  \u003cdiv id=\"df-f770bd8b-35f6-4013-b146-30c839124564\"\u003e\n",
              "    \u003cdiv class=\"colab-df-container\"\u003e\n",
              "      \u003cdiv\u003e\n",
              "\u003cstyle scoped\u003e\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "\u003c/style\u003e\n",
              "\u003ctable border=\"1\" class=\"dataframe\"\u003e\n",
              "  \u003cthead\u003e\n",
              "    \u003ctr style=\"text-align: right;\"\u003e\n",
              "      \u003cth\u003e\u003c/th\u003e\n",
              "      \u003cth\u003earchitecture\u003c/th\u003e\n",
              "      \u003cth\u003edataset\u003c/th\u003e\n",
              "      \u003cth\u003essize\u003c/th\u003e\n",
              "      \u003cth\u003ezca\u003c/th\u003e\n",
              "      \u003cth\u003el\u003c/th\u003e\n",
              "      \u003cth\u003eckpt\u003c/th\u003e\n",
              "      \u003cth\u003etest_acc\u003c/th\u003e\n",
              "      \u003cth\u003etest_mse\u003c/th\u003e\n",
              "      \u003cth\u003eURL\u003c/th\u003e\n",
              "    \u003c/tr\u003e\n",
              "  \u003c/thead\u003e\n",
              "  \u003ctbody\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e0\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar10\u003c/td\u003e\n",
              "      \u003ctd\u003e10\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.260900\u003c/td\u003e\n",
              "      \u003ctd\u003e0.042288\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/cifar10/ConvNet_ssize10_zca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e1\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar10\u003c/td\u003e\n",
              "      \u003ctd\u003e10\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.262900\u003c/td\u003e\n",
              "      \u003ctd\u003e0.042285\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/cifar10/ConvNet_ssize10_nozca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e2\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar10\u003c/td\u003e\n",
              "      \u003ctd\u003e100\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.535900\u003c/td\u003e\n",
              "      \u003ctd\u003e0.034978\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/cifar10/ConvNet_ssize100_zca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e3\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar10\u003c/td\u003e\n",
              "      \u003ctd\u003e100\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.462600\u003c/td\u003e\n",
              "      \u003ctd\u003e0.036484\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/cifar10/ConvNet_ssize100_nozca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e4\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar10\u003c/td\u003e\n",
              "      \u003ctd\u003e500\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.658900\u003c/td\u003e\n",
              "      \u003ctd\u003e0.029552\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/cifar10/ConvNet_ssize500_zca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e5\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar10\u003c/td\u003e\n",
              "      \u003ctd\u003e500\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.573700\u003c/td\u003e\n",
              "      \u003ctd\u003e0.032066\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/cifar10/ConvNet_ssize500_nozca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e6\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar100\u003c/td\u003e\n",
              "      \u003ctd\u003e100\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.238100\u003c/td\u003e\n",
              "      \u003ctd\u003e0.004734\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/cifar100/ConvNet_ssize100_zca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e7\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar100\u003c/td\u003e\n",
              "      \u003ctd\u003e100\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.180700\u003c/td\u003e\n",
              "      \u003ctd\u003e0.004739\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/cifar100/ConvNet_ssize100_nozca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e8\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar100\u003c/td\u003e\n",
              "      \u003ctd\u003e500\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.349300\u003c/td\u003e\n",
              "      \u003ctd\u003e0.004540\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/cifar100/ConvNet_ssize500_zca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e9\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar100\u003c/td\u003e\n",
              "      \u003ctd\u003e500\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.275400\u003c/td\u003e\n",
              "      \u003ctd\u003e0.004560\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/cifar100/ConvNet_ssize500_nozca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e10\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar100\u003c/td\u003e\n",
              "      \u003ctd\u003e1000\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.391700\u003c/td\u003e\n",
              "      \u003ctd\u003e0.004436\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/cifar100/ConvNet_ssize1000_zca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e11\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar100\u003c/td\u003e\n",
              "      \u003ctd\u003e1000\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.312700\u003c/td\u003e\n",
              "      \u003ctd\u003e0.004469\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/cifar100/ConvNet_ssize1000_nozca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e12\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003efashion_mnist\u003c/td\u003e\n",
              "      \u003ctd\u003e10\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.653000\u003c/td\u003e\n",
              "      \u003ctd\u003e0.027842\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/fashion_mnist/ConvNet_ssize10_nozca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e13\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003efashion_mnist\u003c/td\u003e\n",
              "      \u003ctd\u003e100\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.808400\u003c/td\u003e\n",
              "      \u003ctd\u003e0.016468\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/fashion_mnist/ConvNet_ssize100_nozca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e14\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003efashion_mnist\u003c/td\u003e\n",
              "      \u003ctd\u003e500\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.868500\u003c/td\u003e\n",
              "      \u003ctd\u003e0.011954\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/fashion_mnist/ConvNet_ssize500_nozca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e15\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003emnist\u003c/td\u003e\n",
              "      \u003ctd\u003e10\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.733500\u003c/td\u003e\n",
              "      \u003ctd\u003e0.027602\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/mnist/ConvNet_ssize10_nozca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e16\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003emnist\u003c/td\u003e\n",
              "      \u003ctd\u003e100\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.963900\u003c/td\u003e\n",
              "      \u003ctd\u003e0.009460\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/mnist/ConvNet_ssize100_nozca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e17\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003emnist\u003c/td\u003e\n",
              "      \u003ctd\u003e500\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.982800\u003c/td\u003e\n",
              "      \u003ctd\u003e0.005096\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/mnist/ConvNet_ssize500_nozca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e18\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003esvhn_cropped\u003c/td\u003e\n",
              "      \u003ctd\u003e10\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.239052\u003c/td\u003e\n",
              "      \u003ctd\u003e0.043638\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/svhn_cropped/ConvNet_ssize10_zca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e19\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003esvhn_cropped\u003c/td\u003e\n",
              "      \u003ctd\u003e10\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.211048\u003c/td\u003e\n",
              "      \u003ctd\u003e0.043967\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/svhn_cropped/ConvNet_ssize10_nozca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e20\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003esvhn_cropped\u003c/td\u003e\n",
              "      \u003ctd\u003e100\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.527812\u003c/td\u003e\n",
              "      \u003ctd\u003e0.034181\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/svhn_cropped/ConvNet_ssize100_zca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e21\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003esvhn_cropped\u003c/td\u003e\n",
              "      \u003ctd\u003e100\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.400469\u003c/td\u003e\n",
              "      \u003ctd\u003e0.038717\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/svhn_cropped/ConvNet_ssize100_nozca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e22\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003esvhn_cropped\u003c/td\u003e\n",
              "      \u003ctd\u003e500\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.768362\u003c/td\u003e\n",
              "      \u003ctd\u003e0.024390\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/svhn_cropped/ConvNet_ssize500_zca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e23\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet\u003c/td\u003e\n",
              "      \u003ctd\u003esvhn_cropped\u003c/td\u003e\n",
              "      \u003ctd\u003e500\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.692263\u003c/td\u003e\n",
              "      \u003ctd\u003e0.029268\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/svhn_cropped/ConvNet_ssize500_nozca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e24\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet3\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar10\u003c/td\u003e\n",
              "      \u003ctd\u003e10\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.265100\u003c/td\u003e\n",
              "      \u003ctd\u003e0.042152\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/cifar10/ConvNet3_ssize10_zca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e25\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet3\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar10\u003c/td\u003e\n",
              "      \u003ctd\u003e10\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.263800\u003c/td\u003e\n",
              "      \u003ctd\u003e0.042304\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/cifar10/ConvNet3_ssize10_nozca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e26\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet3\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar10\u003c/td\u003e\n",
              "      \u003ctd\u003e100\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.529000\u003c/td\u003e\n",
              "      \u003ctd\u003e0.035220\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/cifar10/ConvNet3_ssize100_zca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e27\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet3\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar10\u003c/td\u003e\n",
              "      \u003ctd\u003e100\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.464400\u003c/td\u003e\n",
              "      \u003ctd\u003e0.036509\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/cifar10/ConvNet3_ssize100_nozca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e28\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet3\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar10\u003c/td\u003e\n",
              "      \u003ctd\u003e500\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.654800\u003c/td\u003e\n",
              "      \u003ctd\u003e0.029904\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/cifar10/ConvNet3_ssize500_zca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e29\u003c/th\u003e\n",
              "      \u003ctd\u003eConvNet3\u003c/td\u003e\n",
              "      \u003ctd\u003ecifar10\u003c/td\u003e\n",
              "      \u003ctd\u003e500\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "      \u003ctd\u003e0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.568900\u003c/td\u003e\n",
              "      \u003ctd\u003e0.032244\u003c/td\u003e\n",
              "      \u003ctd\u003ehttps://storage.cloud.google.com/kip-datasets/ls/cifar10/ConvNet3_ssize500_nozca.npz\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "  \u003c/tbody\u003e\n",
              "\u003c/table\u003e\n",
              "\u003c/div\u003e\n",
              "      \u003cbutton class=\"colab-df-convert\" onclick=\"convertToInteractive('df-f770bd8b-35f6-4013-b146-30c839124564')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\"\u003e\n",
              "        \n",
              "  \u003csvg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\"\u003e\n",
              "    \u003cpath d=\"M0 0h24v24H0V0z\" fill=\"none\"/\u003e\n",
              "    \u003cpath d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/\u003e\u003cpath d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/\u003e\n",
              "  \u003c/svg\u003e\n",
              "      \u003c/button\u003e\n",
              "      \n",
              "  \u003cstyle\u003e\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  \u003c/style\u003e\n",
              "\n",
              "      \u003cscript\u003e\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-f770bd8b-35f6-4013-b146-30c839124564 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-f770bd8b-35f6-4013-b146-30c839124564');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '\u003ca target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb\u003edata table notebook\u003c/a\u003e'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      \u003c/script\u003e\n",
              "    \u003c/div\u003e\n",
              "  \u003c/div\u003e\n",
              "  "
            ],
            "text/plain": [
              "   architecture  ...                                                                                        URL\n",
              "0       ConvNet  ...           https://storage.cloud.google.com/kip-datasets/ls/cifar10/ConvNet_ssize10_zca.npz\n",
              "1       ConvNet  ...         https://storage.cloud.google.com/kip-datasets/ls/cifar10/ConvNet_ssize10_nozca.npz\n",
              "2       ConvNet  ...          https://storage.cloud.google.com/kip-datasets/ls/cifar10/ConvNet_ssize100_zca.npz\n",
              "3       ConvNet  ...        https://storage.cloud.google.com/kip-datasets/ls/cifar10/ConvNet_ssize100_nozca.npz\n",
              "4       ConvNet  ...          https://storage.cloud.google.com/kip-datasets/ls/cifar10/ConvNet_ssize500_zca.npz\n",
              "5       ConvNet  ...        https://storage.cloud.google.com/kip-datasets/ls/cifar10/ConvNet_ssize500_nozca.npz\n",
              "6       ConvNet  ...         https://storage.cloud.google.com/kip-datasets/ls/cifar100/ConvNet_ssize100_zca.npz\n",
              "7       ConvNet  ...       https://storage.cloud.google.com/kip-datasets/ls/cifar100/ConvNet_ssize100_nozca.npz\n",
              "8       ConvNet  ...         https://storage.cloud.google.com/kip-datasets/ls/cifar100/ConvNet_ssize500_zca.npz\n",
              "9       ConvNet  ...       https://storage.cloud.google.com/kip-datasets/ls/cifar100/ConvNet_ssize500_nozca.npz\n",
              "10      ConvNet  ...        https://storage.cloud.google.com/kip-datasets/ls/cifar100/ConvNet_ssize1000_zca.npz\n",
              "11      ConvNet  ...      https://storage.cloud.google.com/kip-datasets/ls/cifar100/ConvNet_ssize1000_nozca.npz\n",
              "12      ConvNet  ...   https://storage.cloud.google.com/kip-datasets/ls/fashion_mnist/ConvNet_ssize10_nozca.npz\n",
              "13      ConvNet  ...  https://storage.cloud.google.com/kip-datasets/ls/fashion_mnist/ConvNet_ssize100_nozca.npz\n",
              "14      ConvNet  ...  https://storage.cloud.google.com/kip-datasets/ls/fashion_mnist/ConvNet_ssize500_nozca.npz\n",
              "15      ConvNet  ...           https://storage.cloud.google.com/kip-datasets/ls/mnist/ConvNet_ssize10_nozca.npz\n",
              "16      ConvNet  ...          https://storage.cloud.google.com/kip-datasets/ls/mnist/ConvNet_ssize100_nozca.npz\n",
              "17      ConvNet  ...          https://storage.cloud.google.com/kip-datasets/ls/mnist/ConvNet_ssize500_nozca.npz\n",
              "18      ConvNet  ...      https://storage.cloud.google.com/kip-datasets/ls/svhn_cropped/ConvNet_ssize10_zca.npz\n",
              "19      ConvNet  ...    https://storage.cloud.google.com/kip-datasets/ls/svhn_cropped/ConvNet_ssize10_nozca.npz\n",
              "20      ConvNet  ...     https://storage.cloud.google.com/kip-datasets/ls/svhn_cropped/ConvNet_ssize100_zca.npz\n",
              "21      ConvNet  ...   https://storage.cloud.google.com/kip-datasets/ls/svhn_cropped/ConvNet_ssize100_nozca.npz\n",
              "22      ConvNet  ...     https://storage.cloud.google.com/kip-datasets/ls/svhn_cropped/ConvNet_ssize500_zca.npz\n",
              "23      ConvNet  ...   https://storage.cloud.google.com/kip-datasets/ls/svhn_cropped/ConvNet_ssize500_nozca.npz\n",
              "24     ConvNet3  ...          https://storage.cloud.google.com/kip-datasets/ls/cifar10/ConvNet3_ssize10_zca.npz\n",
              "25     ConvNet3  ...        https://storage.cloud.google.com/kip-datasets/ls/cifar10/ConvNet3_ssize10_nozca.npz\n",
              "26     ConvNet3  ...         https://storage.cloud.google.com/kip-datasets/ls/cifar10/ConvNet3_ssize100_zca.npz\n",
              "27     ConvNet3  ...       https://storage.cloud.google.com/kip-datasets/ls/cifar10/ConvNet3_ssize100_nozca.npz\n",
              "28     ConvNet3  ...         https://storage.cloud.google.com/kip-datasets/ls/cifar10/ConvNet3_ssize500_zca.npz\n",
              "29     ConvNet3  ...       https://storage.cloud.google.com/kip-datasets/ls/cifar10/ConvNet3_ssize500_nozca.npz\n",
              "\n",
              "[30 rows x 9 columns]"
            ]
          },
          "execution_count": 4,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "with tf.io.gfile.GFile('gs://kip-datasets/ls/ls.csv') as f:\n",
        "  df = pd.read_csv(f)\n",
        "df"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SvCG3dVsnDzG"
      },
      "source": [
        "# Load Checkpoint"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "executionInfo": {
          "elapsed": 387,
          "status": "ok",
          "timestamp": 1642197643404,
          "user": {
            "displayName": "",
            "photoUrl": "",
            "userId": ""
          },
          "user_tz": 480
        },
        "id": "rJuqmje8LvtF"
      },
      "outputs": [],
      "source": [
        "with tf.io.gfile.GFile('gs://kip-datasets/kip/cifar10/ConvNet_ssize100_nozca_l_noaug_ckpt1000.npz', 'rb') as f:\n",
        "  npz = np.load(f)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "executionInfo": {
          "elapsed": 55,
          "status": "ok",
          "timestamp": 1642197644193,
          "user": {
            "displayName": "",
            "photoUrl": "",
            "userId": ""
          },
          "user_tz": 480
        },
        "id": "ErTDjzJ7Lxhc",
        "outputId": "97eb98d5-ccf9-4869-b1d4-2773c1dd180c"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "['images', 'labels']"
            ]
          },
          "execution_count": 6,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "list(npz.keys())"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "name": "dataset.ipynb",
      "provenance": [
        {
          "file_id": "1RpXJA8-t2aDTq3XuQWnTD2OE9qxct721",
          "timestamp": 1627527061297
        }
      ]
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
